DOCUMENT BESOHE 

TH 003 728 

Ford, Valeria A. 

Everything You Wanted To Know About Test- Wiseness. 

Aug 73 

30p. 

MF-S0.75 HC-S1.85 PLUS POSTAGE 

Guessing (Tests) ; *Li terature Reviews ; Multiple 
Choice Tests; *Response Style (Tests) ; Testing 
Problems ; ♦Test Wiseness 



The purpose of this paper is to acguaint the reader 
with the topic of test- wiseness. The first section of this paper 
presents a series of multiple-choice items. The reader is asked to 
respond to them and is encouraged to read carefully the remainder of 
this paper for techniques which could improve test-taking 
performance. The next section defines and elaborates on the term 
test-wiseness as it relates to other psychometric terminology. Tfcen, 
an attempt is made to distinguish test-wiseness from several other 
concepts frequently used indiscriminately. The third section, 
entitled "Selected Aspects of Test-wiseness," outlines a set of 
behaviors that a test-wise examinee should employ as he takes a test. 
Empirical research to support the content of the outline is 
elaborated in detail in the succeeding section. Finally, a rationale 
for the use of a test-wiseness learning program to be used in the 
school is presented, and several possible procedures as to how such a 
program could be implemented are suggested. (A.uthor/BB) 
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An Overview 



The purpose of this paper is to acquaint the reader to the Luac of test- 
viseness. Test-wiseness , as an issue in educational and psychological measure- 
ment, has been investigated rather extensively and independently by numerous 
researchers, however, no attempt, familiar to this writer, has been irade to 
organize the research into a systematic presentation. In acquainting the reader 
to the topic, it is hoped that an understanding of one of the important influences 
effecting test performance will be acquired. 

s 

The first section of this paper presents a series of multiple-choice items. 
The reader is asked to respond to them anc is encouraged to read carefully the 
remainder of this paper for techniques whioh could improve test-taking performance. 

The next section defines and elaborates upon the term test-w i sencss as it 
relates to other psychometric terminology. Then, an attempt is made to distin- 
guish test-wiseness from several other concepts frequently u^cd indiscriminately. 

The third section, entitled "Selected Aspects of Test-Wiseness 11 outlines 
a set of behaviors that a test-wise examinee should employ as he takes a test. 
Empirical research to support the content of the outline is elaborated in detail 
in the succeeding section. 

Finally, a rationale for the use cf a test-wiseness learning program to be 
used in the school is presented and several possible procedures as to how such 
a program could be implemented are suggested. 

Your Test-Wiseness Quotient 

This is a test of test "viseness." Answer each <;uestion after you have decided 
on the response you feel will earn you the highest score. Each question can be 
answered by employing several test-wiseness techniques. 



The typical "potassium potential" across a nervc-axon membrane is 

A. -SO m V 

B. -12d m V 

C. -35 m V 

D. -60 in V 

The Municipal Corporation Bills, enacted by Parliament in 1835 

A. Guaranteed the rights and freedoms of large urban industries 

B. eliminated "rotten boroughs" by redisricting urban areas 

C. provided a plan of government for most areas except London and 
romc small towns 

D. gave Parliament the right to interfere with private industry 
The Fisher Act, legislated by Parliament 

A. established a national system of public education with tax-supported, 

public elementary and secondary schools 

B. reformed "rotten boroughs" 

C« reduced the power of the House of Lords 
D. created a system of socialized medicine 

Inhibition of the motor neurons may be brought about by 

A. an anodic increase in the Ner^st potential 

B. elicitation of a synaptic potential across the postsynaptic Membrane 

C. blockage of acetylcholine at the synaptic junction by chemical inactivation 
D« intramuscular administration of physiological saline solution 

"The Salad" is a fairy tale by: 

A. Percy Bysshe Shelly 

B. the Brothers Grimm 

C. George Meredith 
D« John Donne 

In his first message to Congress, President Theodore Roosevelt: 

A. demanded greater control of large corporations by the federal government 

B. avoided the tariff question as politically too controversial 

C. called for the reaffirmation of an economic policy with strongly favored 
free enterprise 

D. vehemently advocated a higher tax schedule on imports 

The Compromise between Democrats and Republicans after the election of 1876 
resulted in 

A, federal aid to Southern railroads 

B, a higher federal tariff to assise the automobile industry 
C« the entrance of Texas into the Union as a slave state 

D, a treaty with Joseph Sta] in 

E« the Fugitive Slave Law 
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In the autonomic division 

A. the cell bodies of the motor neurons are located exclusively within the 
brain and spinal cord 

B. the cell bodies of the motor neurons may be founc] near the organs supplied 

C. the fibers cannot inhibit the organ which is supplied 

D. the organs supplied may be inhibited as well as stimulated by the fibers 

The population of Frankton is more than 

A. 50 million 

B. 60 million 

C. 70 million 

D. 80 million 

10. When Beston crystals are added to water 

A. heat is given off 

B. the temperature of the solution rises 

C. the solution turns blue 

D. the container becomes warmer 

Do not be discourged by these items. By reading the remainder of this paper, 
you will gain a greater understanding of the topic of test-wiseness and, hopefully 
your sophistication in test-taking techniques vill he improved. 



TEST-WISENESS: THE STATE OF THE ART 



Test-wiseness and test score variance 



The level of attainment of a group or an individual can be measured by a 
variety of objective techniques — the best known among them being standardized 
tests. Most of the existing objective tests are composed of items which sample 
a learner's behavior at the precise moment of testing. The score on these tests 
is the outcome cf the learner's total experience, which in turr is a result of 
(1) a combination of such factors as his overall innate ability, (2) his general 
environment, and (3) his special knowledge in the subject matter tapped by the 
test. In addition, certain studies have noted the influence of ,! tes t~wiseness" 
on test scores (Dunn & Goldstein, 1959; Millman & Setijadi, 1966; Vernon, 1962). 
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Test-wiseness has been defined as the ability to manifest test-taking skills 
which utilize the characteristics and formats of a tost and/or test-taking 
situation in order to receive a score commensurate with the abilities being 
measured (Oakland & Weilert, 1971)- Other names for this trait have been 
offered such as "test sophistication" (Krickson, 1972) or "test-wisdom 11 (Preston, 
1964). 

If it could be assumed that all but inborn factors were equal for all 
learner's (e.g., environment and/or training), the test scores obtained could 
be considered as representing the learner's true ability. If the nature of the 
test material is such that some students are at an advantage from having past 
experiences in test-taking, then .some inaccuracy in measurement can occur. 

A test-wise examinee can be expected to obtain a higher score on an 
aptitude, achievement, or teacher-made test, than an equally competent examinee 
who lacks test sophistication. Minority spokesmen argue that part of the reason 
for lower scores for their children results from a lack of familiarity with 
the "tricks" of taking a test. If, in fact, test scores for children of 
certain backgrounds are lower as a result of a lack of familiarity with some 
non-content factor (e.g., not knowing when to guess^ misunderstanding instruc- 
tions), then the argument may be valid. 

Test constructors have attempted to reduce the variance in test scores 
resulting from degrees of test sophistication by writing clearer directions, 
eliminating specific determiners (e.g., often, seldom, always, must, etc. ); 
however, various authors (Ebel, 1965) have conjectured that test-wiseness 
produces an additional source of test-reated influence in educational test 
scores beyond that due to item content or random error. In an effort to 
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assure that knowledgeable students are not unduly penalized by poor examination 
skills, considerations have been given to training individuals who lack test 
sophistication. Wood ley (1972) considers test-wiseness to be a cognitive 
factor, one which is measurable and subject to change either through specific 
test experience or training in a test-taking strategy. Consequently, rather 
than viewing test-wiseness as insignificant or undesirable, a growing number 
of recent investigations, suggests that people should be given training in 
"how to take tests. 11 

Test-wiseness and subject matter coaching 

Several terms have been used to express the, many kinds of short-term inter- 
vention programs which have been designed to effect score gains of students taking 
tests intended to measure school achievement and general aptitude. Numerous 
research efforts across a whole range of approaches have attempted to cause some 
changes in short periods of time. Differences in the kind of training and/or 
content of instruction incorporated in the intervention program or experimental 
treatment constitute a workable distinction between the terms. In order to 
facilitate discussion for the remainder of this paper, the two terms will be 
di f f erentia ted . 

Several studies cited in the research literature have focused on substantive 
test content and have come to be classified under the generic of "coaching studies;" 
intensive tutoring by a commercial tutoring school in vocabulary, reading, concept 
mastery, and problem-solving (Whitla, 1962); accelerated reading (Coffman & Tarry, 
1967); and commercial coaching with questions similar to those on SAT (Frankel, 
1960). Findings by those who have explored the effects of coaching intervention 
on test scores (Dyer, 1953; French, 1955; French and Dear, 1959) have, historically, 
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yicl.ded negligible results. Anastasi (1968) nh»kcs the point that "the more 
closely instruction is restricted to specific test content, the less likely is 
improvement to extend to criterion performance." 

In what is commonly known as the Tennessee Study, Roberts & Oppenheim (1966) 
developed a specially nrepared linear programed material, designed to foster an 
analytic approach to the tasks required on college admissions in the verbal & 
mathematical areas. Instruction in either of the two areas was given to 11th 
grade students. Alternate forms of the PSAT, administered as pre- and post-tests 
were used to evaluate the effectiveness of the special instruction. While there 
were statistically significant differences in gain scores between some of the 
experimental groups, and their controls, the outcome of the study, like the outcome 
of earlier studies, was generally negative. The authors concluded that "the 
magnitude of the gains observed in the experimental groups were so small that it 
does not seem reasonable to expect that similar short term instruction given on 
a wide scale would be of significant benefit," The authors caution that the 
seven and one-half hour of instruction is less than what has been given in other 
coaching studies or at commerical coaching schools. 

Based upon several coaching studies investigated by the College Entrance 
Examination Board, the trustees of the CEEB issued a detailed statement assert- 
ing that the general score gains directly attributable to coaching generally 
amount to fewer than 10 points, "a difference of such small magnitude on the SAT 
score scale of 200 to 800 that it is unreasonable to expect it to affect admissions 
decisions (College Board, 1965, p, 4)," 

Recently, an investigation of the effects of special coaching on mathe- 
matics items on GRE test performance (Pike & Evans, 1972) has met with some 
success, however, the method cf instruction has been criticized (Flaugher, 1972) 
because of the excessive length and intensity of the program. 



In contrast to coaching studies, test-wi senoss studios have focused upon 
instruction in general test-taking strategies (Moore, 1968), instruction in 
strategics specific to items grouped by item format (Moure, 1971), practice 
with the kinds of tests the examinee will encounter in the testing situation 
(Droege, 1966) and motivational activities prior to testing (Onvig, 1971). 
Tes t-wiseness is generally independent of the examinees knowledge of the 
trait or the subject matter for which the items nre supposedly measures. 
Thus, in test-wiseness studies, instruction in subject matter material has 
been avoided. The primary concern has been to concentrate on the cognitive 
skills prerequisite to successful test taking. Since 1965, a variety of 
learning experiences designed to facilitate test-wiseness or specif i^ skills 
related to test-wiseness have been developed. In most cases, the studies 
have indicated that the levels of test-wiseness can be increased through 
instruction (Moore, Schutz & Baker, 1966; Wahlstrom & Boerma, 1968; Oakland 
& Weilart , 1971) . 

Effects of Practice on Test 

Familiarity with the format used in standardized tests pertain to a person 1 s 
test wiseness. Millman and Setijadi's (1966) study demonstrated the disadvantage 
under which students operate when taking a test with an unfamiliar format. Compari- 
sons were made of the performance of Indonesian and American students on tests 
using open-ended and multiple-choice questions involving arithmetic computation 
and vocabulary. The Indonesian students did relatively less well on the multiple- 
choice items than they did on the open-ended questions. This differential perfor- 
mance presumably was due to their having had no prior experience with multiple-choice 
items. 
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It has often been assumed that the experience of taking aptitude tests often 
loads to an increase in test scores upon retesting. Syslemated study of the effect 
of practice supports this assumptTon. 

Kreit (1967) examined the effects of test practice on the acquisition of test- 
taking skills of third grade pupils. Four different group intelligence tests were 
administered to the experienced group, while the control group received only pre- 
and post-tests. Increases in test-taking skills were acquired by the experimental 
group, presumably from increased exposure to a variety of tests. 

Droege (1966) investigated the long-range effects cf practice on the General 
Aptitude Test Battery (GATB) , with an alternate form after 1 year, 2 years, and 
3 years. The major findings revealed that increases in aptitude means of scores 
between initial testing and retesting show that all of the increases were 
significant, and that the effects of practice appear to be operating for all 
subsamples even when the interval between testings is as long as 3 years. 

Familiarization with Item Types 

In a study previously cited (Roberts & Oppenheim, 1966) , materials consisting 
of "courses" of programed instruction in five areas (four devoted to specific 
item types and a fifth to test-taking in general) were developed. Programs for 

m 

the various item types (sentence completion, antonyn, analogies, reading compre- 
hension) and to create an analytic approach to test questions. Practice questions 
with the various item types were provided. Statistical significance was reported 
for post test scores of experimental and control groups, however, this difference 
was attributable to a substantial droo in the post-test scores of the control 
group. 
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Invost igat ions into the various aspects of tcst-widemess were basically 
a result of a descriptive paper written by Millman, Bi Sop and Cbcl (1965), 
which provided a conceptual framework for the topic. Based upon empirical studies 
froir. the Last fifteen years, an attempt is made to identify useful strategies for 
taking tests which coincide with/or expand the informal "tes t -wiscness principles" 
outlined by MilJman ot .il. The following outline was developed from a variety 
of learning programs related to test-wiseness . 

Selected Aspects of Test Wiseness 



1. Instructions in general test-taking strategies 

A . Follow directions 

1. listen carefully over an expanded period of time [standardized 
tests contain important directions that arc read orally.] 

2. pay close attention to directions concerning allotted time and 
penalty cont ingences . 

3. be sure to understand directions [if directions are not under- 
stood, ask the examiner immediately for clarification]. 

B. Properly mark responses (item response mode) 

1. understand the proper way to mark responses [if the appropriate 
way to mark responses is not clear, immediately ask the examiner]. 

2. remember that different parts of standardized tests require 
the ability to make appropriate responses, as required, for 
each of the different parts of the test. 

3. remember to select the correct c^lternative , identify the 
alternatives' code and item number, transfer this information 
from the test booklet (if necessary) to the answer sheet. 

C . Use time* efficiently 

1. first answer questions of vhich you are sure. 

2. then do the more difficult and time-consuming questions. 

3. check answers during any remaining time to assess correctness 
and avoid careless mistakes. 

4. if time permits, return to those questions previously omitted. 

5. don't ponder over alternatives. 
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^ * Learn when and when not to gues_s 

1. guess only after an honest attempt has been made to answer the 
quest ion. 

2. guess , especially if one or more wrong answers can be identified. 

3. if there is no penalty for guessing, answer every question since 
any question answered correctly will add to rhe total number 
correct . 

A. if the test is a power test, with a penalty for guessing, omit 
any question for which at least one choice cannot be eliminated. 

E . Miscellan e ou r> t i p s 

1. do not lies it ate to change an answer, if you feel you should, 
since the percentage of wrong to right revisions tends to exceed 
right to wrong. 

2. examine carefully all possible responses before attempting to 
choose the correct answer . 

3. learn to concentrate attention to test-relevant variable while 
directing attention away from sel f -eval ua t ive ruminations. 

a. don't worry about your performance or how well others are 
do ing. 

b. don't ruminate over alternatives. 

s 

II. Instructions in strategies specific to item format 

A. Answering^ analogy questions 

1. learn to recognize common kinds of relationships presented in 
analogy questions [purpose, cause effect, part-to-whole, part- 
to-part, action to object, object-to-action, symonym, antonym, 
degree, characteristic, sequence, grammatical, numerical, 
association. ] 

2. plan a strategy for answering analogy questions - learn to 
form an immediate associate relatedness among the words in an 
analogy item. 

B. Answering multiple-choice questions 

1. eliminate options known to be incorrect and choose from the 
remaining alternatives. 

2. eliminate similar options i.e., options which imply the correct- 
ness of each other. 

3. eliminate those options which include specific determiners. 

4. select the optioft which resembles an aspect of the stem, such 
that a name or phrase is repeated in both the stem and alterna- 
tive . 

5. look for correct alternatives which are longer and in some cases 
more precise and specific in meaning than other alternatives. 

6. be alert to grammatical clues in the stem. 

7 • watch for overlapping distractors in which "the truth of one 
alternatives implies the correctness of several others. 

8. beware of one item "giving away' 1 the answer to another item 
oecuring in a latter part of the test. 

As stated earlier, the preceding outline is based upon recent f indings in 
test wiseness. Research literature supporting the content of the outline will 



be summarized. 
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A RKV1KW OF THE LITERATURE 

Research in Support of Changing Answers 

Often students are heard to ask whether or not they should reconsider 
and possibly change their response to a test item. Writers on the topic of test- 
taking are, for the most part, in agreement concerning the advisability of answer 
changing. Millman et. al. suggests that the tendency to evaluate and judiciously 
change one's item response is a basic aspect of tes t-wiseness . Conclusions from 
recent research (Bath, 1967; Reile and Briggs, 1952; Stanley, 1971; Cop^land, 

1972) suggest that it is apparently advisable to change one's responses, since 

i 

the typical result is an improvement in test scores. 

In order to ascertain the effect on test grades of changing answers, Lynch 
and Smith (1972) administered a multiple choice test to 178 undergraduate students. 
Of the 294 response changes they identified, 68% were from wrong to right as 
compared with 32% for the reverse* This finding underscores earlier studies 
(Berrein, 1939; Lowe .and Crawford, 1939), however, the authors reported that 

repeated changes on j the same item did not improve a person's score. 

/ 

Research on Guessing Strategies / 

When students are told they will be given a multiple-choice test, they 
sometimes ask if they will be penalized for guessing. Most often students are 
advised not to guess on standardized multiple-choice tests. Earlier evidence 
(Sheriffs and Boomer, 1954) has indicated that when examinees 1 are requested 
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to respond to \ terns previously omitted, their mean test score will increase. 
However, recent Experiments investigating the effect of guessing on test scores 
report somewhat conflicting findings, 

Taylor (1966) studied three different instruction groups ("do not answer 
unless certain; 11 "do las well as you can;" "answer all questions") and found no 
differences in mathematics scores due to instructions. On the other hand, 
Hammerton (1965) found \that when examinees were instructed to answer all ques- 
tions y they scored higner on vocabulary tests than when instructed to leave out 

questions if not corrxident of the answer. 

/ 

The finding of\(^cliael e_t. al . (1963) indicated that the average l^vel of 
performance increased ./n&ia guessing was encouraged and decreased with instructions 
advising against guessing. Another researcher (fibel, 1968) concludes from his 
study with four separate > true-false tests that "students seeking highest scores 
on a test are well advised to answer all questions even when the usual correction 
is applied (their blind gtfesses to true-false items tend to be correct more than 
half of the time.) 1 Slakter (1968) sought to determine which of the following 
str ajie^ ies would result in a higher test score: (a) you will receive one point for 
each correct answer, lose 1/3 of a point for eacl incorrect answer, and neither 
receive nor lose points for questions you omit: usual guessing strategy (b) answer 
every question, whether you are sure of the answer or not: high guessing strategy. 
With these conventional directions, the results demonstrate th,<t the "high guessing" 
group averaged five points higher than the "usual guessing" group, which indicated 
that the guessing effect was significant. 



With the exception of the Taylor study, which has been criticized on method- 
ological bases (Slakter, 1968), it appears that an examinee would tend to increase 
his test score by answering all items, even though a penalty for incorrect responses 
may be in effect. 
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In a somewhat different approach to research on guessing, Moore, Shultz 
and Raker (1 ( >66) developed a programmed text designed to help students develop 
optimal guessing strategies for responding to both speed and power tests. Several 
sample test directions gave the learner experience in identifying the important 
test contingencies. Various item sets provide guided practice in utilizing 
efficient procedures with resrect to "guessing 11 and pacing of responses. 



Penalty 


Power 


Speed 


High 


Low 


for Guessing 


Test 


Test 


Guessing 


Guessing 


Yes 


Yes 


No 


No 


Yes 


No 


Yes 


No 


Yes 


No 


No 


No 


Yes 


Yes 


No 


Yes 


No 


Yes 


No 


Yes 



Table 1 reviews penalty and time characteristics, in addition to the optimal 
guess strategy under each condition. The data for the study consisted of 
guessing scores on four criterion tests in which directions regarding penalty and 
time contingencies varied (e.g. power test with no penalty for guessing, speed test 
with penalty for guessing). Research with the text showed a statistically signifi- 
cant difference between experimental and control groups in terms of guessing 
performance. 

In another research study addressing this problem, Callenbach (1973) provided 
instruction to students regarding guessing. Instruction focusing on guessing 
stressed that students should (1) first answer questions to which one is sure and 
(2) guess only after an honest attempt has been made to answer the questions. 
Although these guessing instructions were only a part of the total instructional 
program in content - independent test-taking techniques, the overall results revenled 
a significant difference between experimental and control groups on the Stanford 
Reading Test. 
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Ro search Rc^l ^ t c-d to Solving Analogy Problems 

Analogy items of the form accompanying the Miller Analogy Test (MAT): 

light: dark: pleasure: 

a. picnic b. day c. pain d. night 

are an integral part of the testing many levels: high school, college, and graduate 
school. Some research suggests the possibility that the task of completing analogy 
questions, in nany instances, is unique to many examinees, and that having exposure 
to such problems tends to facilitate performance. Colardarci (1960) reported that 
MAT retest performance was substantially higher than performance on aa initial test. 
Moore (1966) correlated an inverse relationship between the time needed to complete 
sample questions and MAT scores. He suggests that not understanding the analogy 
task night pose a problem for many students. In a later study (Moore, 1971) the 
analogy test iticm was viewed as having certain characteristics that could be system- 
atically taught:. He hypothesized that examinees who were taught about the analogy 
item would tJe nore able to deal with the relationships implied by the subject matter. 
Materials consisted of instruction on two aspects of test-wiseness in an analogy 
situation: question format, kinds of analogies, and use of the colons. The analysis 
showed a significant effect for the treatment factor. The findings of this study, 
together with others in the area of test~wiseness , suggests the possibility of 
improving the performance of examinees by brief instruction. 

How to Approach Multiple-Choice Items 

Not only have analogy items been considered for use in instructional programs 
in test~wiseness , considerable attention has been devoted to multiple-choice questions. 
Slakter, Koehler, and Hampton (1970) developed a programmed text to teach certain 
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sclected aspects of test wiseness to 5th through 11th grades: (1) stem options, 
(2) absurd-options, (3) similar options, and (4) specific determiners. The 
following are examples of the Mnds of items ns^d to instruct students in these 
four basic types of clues. 

1. stem-options 

The revolution of the Four Hundred took place 

(1) after the unfortunate and long-lamented death of Theramenes 

(2) after the Sicilian expedition by the four hundred swordsmen of Alcibiades. 

(3) before the desperately sought peace of Nicias. 

(4) after the death of Alcibiades. 

2. Absurd-options 

The state vith the highest total population in the Unites States in 1950 was 

(1^ Chicago 

(2) Canada 

<3) New York 

r c) Washington, D.C. 

3. Similar options 

2 

The line that is normal to the cure y - x + 2x - 3 at the point (1,0) intersects 
the curve at one other point. What is the abscissa of this other point? 

(1) -3.25 

(2) -18/4 

(3) -4.50 
'4) -4 1/2 

4. Specific determiners 

French diplomacy between 1789 and 1815 was 

(1) completely unseccessful in establishing better trade relations 

(2) to find some success in spreading revolutionary ideas to the rest of 
Europe 

(3) totally committed to the preservation of the monarchial form of government 
in Europe 

(4) absolutely successful in winning sympathy for Napoleon 
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Wahlstrom and Boersma (1968) conclude that the teaching of to^t-wise techniques 
led to improved performance on multiple-choice tests among the ninth graders they 
studied. Students were assigned to experimental, control, and placebo groups. The 
treatments were, respectively; instruction in principles of tf^st -wiseness (a modified 
version of the principles of Mi 11 man et ■ al . , 1966); watching television; and discus- 
sion of occupational information. Each group received pre- and post-tests in social 
studies which employed either "good" or "poor" multiple-choice items in an attempt 
to evaluate the effects of test-wiseness in objective examinations. Differences 
between pre- and post-test mean scores for control and placebo groups were all 
non-significant, however, the experimental group scored significantly higher than 
its control group. 

In a study similar in design and methodology to Slaker et. al. (1970), a program 
designed specifically for use by adults was aimed at improving the general test-taking 
skills of candidates for the professional designation of Chartered Life Underwriters 
(Woodley, 1973). In order to determine the actual level of test wiseness in this 
population, a test was designed to measure whether or not the examinee could arrive 
at the appropriate answer to an item by recognizing and eliminating (1) similar 
options; and (2) absurd options, and (3) selecting an option which has a logical 
relationship with the stem. 

Instructional packages, one for each of the test wiseness strategies taught, was 
used as the intervention for the experimental group. The test wiseness scale described 
earlier served as the pre- and post-test (internal criterion). All candidates, also 
took the Chartered Life Underwriter examination (external criterion)- The findings 
showed that training in test wiseness produced the desired effect on the internal 
criterion, but did not on the external criterion. 
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Gibb (1964) was also concerned with cues found in multiple choice items and 
characterized test-wiseness as a "secondary cue response . " He administered a 

preliminary assessment of test-wiseness to a group of graders, which included 

seven types of secondary cues: 

(1) sound, word or phrase repetitions between stem and correct alternative 

(2) incorrect alternatives that are grossly unrelated or bear an absurd 
relationship to the stem 

(3) inclusion of words such as '"all," "none," "nerves," etc. in the phrasing 
of incorrect alternatives 

(4) correct alternatives that are clearly more precise in meaning, more 
fully qualifying, or more specific than the incorrect alternatives 

(5) correct alternatives that are visibly longer than incorrect alternatives 

(6) grammatical cues, such as differences in number or tense of verbs, nouns, 
or the indefinite article between stem and alternatives 

(7) one item "giving away" the answer to another item occurring in a different 
part of the test 

An experimental group was given brief training in the nature and recognition 
of these secondary cues. Following training, the experimental group and its control 
were given a 70 item, multiple-choice instrument of test-wiseness. Results showed 
that the trained group profited markedly from its control, and its performance was 
significantly higher. He also indicated that performance on his items was not 
determined by knowledge of the subject matter. 

Based upon a survey of achievement batteries commonly used in the primary grades 
and from a careful analysis of the Stanford Reading Test, primary I, Callenbach (1972) 
constructed experimental materials for instruction and practice in (1) following 
orally administered directions, (2) response marking, (3) using time efficiently, and 
(4) guessing. Instructions stressed that: (1) standardized tests contain important 
directions that are read orally; (2) in order to answer questions correctly, the 
student must listen intently over an extended period of time; (3) in order to 
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answcr the questions correctly, students must understand the directions; (4) different 
parts of standardized tests require the ability to make appropriate responses, as 
required for each of the different parts of the test; (5) when the student do-^s not 
understand the directions, or the proper way to mark responses, he should ask for 
clarification; (6) students should first answer questions he knows answers to; 
(7) difficult and time consuming questions should be answered last; (8) answers 
should be checked during any remaining time to assess correctness and to avoid care- 
less mistakes. 

The standardized reading test scores of test-naive students can be significantly 
raised through instruction and practice in content-independent test-taking techniques. 

Training Students to Be Better Test Takers : Motivational Aspects 

Omvig (1971) investigated the effects of motivational and teaching activities 
carried out prior to testing on test performance. Experimental students were given 
individual counseling on their past record of standardized achievement testing. A 
brief discussion on test taking included (a) why the test was administered, (b) the 
importance of following test directions, (c) marking the answer sheet correctly, 
(d) determining possible correct responses on multiple choice items before marking the 
desired response, (e) factors involved in scoring the test, and (f) an explanation of 
test norms and percentile ranks. When test results (Iowa Tests of Educational Develop- 
ment) for the experimental and control groups were compared, the experimental group 
achieved higher scores on all subtests. However, only subtests dealing with profi- 
ciency in English and mathematics were significantly higher. Although the results 
are not overwhelmingly conclusive, the author contends that improved test results 
are obtainable when students become personally involved in, motivated by, and interested 
in the testing program. 
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Test- Wi seness With Young Childre n 

Most of the previous studies on test-wi seness have dealt with learners in 
upper elementary grades or above. Oakland (1972) attempted to examine certain 
variables which apply to test-wi seness factors in assessing preschool and primary- 
grade children. His intent was to familiarize children to tYz format and language 
use-l in standardized tests. The initial phase of the study identified specific 
abilities which appeared to be prerequisites to taking various standardized readiness 
tests (e.g. Metropolitan Readiness Test, Lee-Clark Reading Readiness Test). Having 
identified these abilities, curricular materials were designed to facilitate the 
development of these abilities. 

The test-wiseness materials utilized several different formats in order to 
provide children with an opportunity to learn to respond to a variety of different 
tasks. No attempt was made to increase content area knowledge. The materials do 
the following: 

1 . Begin with few items and options per page and gradually 
increase them in number until the page is similar in 
appearance to an actual test page. 

2. Provide practice in working in columns and rows. 

3. Teach the concepts right , left , up , down , opposite , 
most like , same , and different * 

4. Gradually increase the number of options per frame from 
two through five. 

5. Encourage children to examine carefully all possible 
responses before choosing the correct one. 

6. Progress from big pictures and words with few on a 

page to small pictures and words with several on a page, 
again until the final page is similar to an actual test 
page. 

7. Provide practice in putting a mark directly on, directly 
under, or in the circle under the correct response. 
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8. Teach the children that biggest can mean the most 
and that a pencil may be called a marker. 

9. Use both dotted lines and heavy black lines to 
separate the criterion from the options. 

10. Refer to each page as a test in order to get 
children accustomed to the word. 

11. Encourage children to ask questions if they do 
not understand the test directions. 

12. Teach children to use a marker for keeping their 
places as they progress on each png6. 

13. Gradually increase the length of time children 
are encouraged to remain task oriented . 



The Metropolitan Readiness Test was 
materials. Results revealed significant 
and control groups for total scores, and 



used to evaluate the effectiveness of the 
group differences between the experimental 
the "Matching 11 sub-test. 



In addition to the various studies that have been reviewed in ^his paper, 
other attmepts have been made to investigate aspects of test-wiseness in several 
different approaches. Several of these attempts ;ill be described below. 

Non-experimental At tempts at Test-Wiseness Programs 

Insomuch as disadvantaged persons tend to have fairly restricted experience 
in taking employment and similar aptitude, intelligence and job knowledge tests, 
it was reasoned that the lack of such exposure leaves them without test-taking skills. 
The U. S. Department of Labor Manpower experimental and demonstration projects have 
explored several approaches in the pretesting preparation of out-of -school and 
out-of-work youth and adults. To develop positive attitudes toward testing and to 
build test-taking confidence, Seller (1971) reports that Labor Departments projects 
found the following of value: 

1. fully explain why tests are given and how they can help 
the individual taking them. 

O 
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2. allow the applicant to have a look at sample 
tests and lake practice tests without time limits 

3. answer questions about tests and explain errors 
made on practice tests 

4. expose applicants to non-threatening test settings 
(test center/room environment) 

5. permit applicants to test each other to become 
familiar with tests and to understand better why 
they are required 

Seiler also describes an E & D project in Tennessee that found it useful to 
give instructions on how to choose answers; when to giuss on true-false, multiple- 
choice, and matching type tests; the advisability of answering the easier questions 
first; and the benefit of using the process of elimination to find correct answers. 
Although Seiler presents no evidence on the effectiveness of these projects, he 
maintains that pretesting orientation should be an extended experience in which the 
applicant must develop skills in the specific types of test directions, questions 
and content found in formal testing. 

Milhalka (1970) reported that the Ohio Bureau of Employment Services developed 
a structured pretest orientation program which included group discussion on reasons 
for testing, kinds of tests, scoring procedures, use of separate answer sheets, 
and test-taking hints. Short simple exercises provided practice with a variety of 
common tests found in industry, government, and education. 

Finally. Erickson (1972) from careful observation as a reading teacher suggested 
techniques and procedures appropriate for providing students with test sophistication 
in answering control thought, detail, context, and inference type reading questions. 
He makes the suggestion that test sophistication skills need to be taught as an 
integral part of developmental, corrective, and remedial reading programs. 
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lmplemcn t ii v f ; a To.U-Wiscness Program i nto tin- S chools 

Tests have been used increasingly in recent years by government, industry, 
and educational institutions in making selection, placement, and guidance decisions. 
But the increasing use of tests has been accompanied by an increasing flow of 
criticism directed toward test producers and consumers. The fairness of tests to 
underprivi lodged and minority groups, the heavy relianced on tests in making 
decisions, and the rigid and inflexible classification all to often placed on an 
individual are three of the most frequently expressed concerns stemming from the 
testing revolution. 

There is no doubt that much can and should be done by test producers to 
improve their instruments, and by consumers of tests to accurately interpret and 
use test results. More importantly, however, is the need for testing services to 
do-emphasize the use of norms, percentile, etc., and for counselors, teachers, 
employers, and laymen to get a clearer understanding of what a person's test 
score represents. It should not be enough to allow an individual's test score 
to merely be viewed in terms of norms, scales, and other technical considerations, 
which in fact are only a small part of what a test score should reflect. As 
Dyer (1971) once observed: 

n We have been so concerned with using test scores to 
classify students in one way or another or to predict 
how they will make out, academically or otherwise, 
that we have, I think, paid far too little attention 
to the details of behavior that a score presumably 
summarizes . 11 
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An individual's test score is a composite measure reflecting both ability and 
experience and as such is influenced by such environmental factors as the educa- 
tional process. When combined with information about an individual's experiential 
background, test scores could facilitate a wide array of educational experiences 
for maximizing individual development. 

From the topic developed in the preceding pagey , test wiseness appears to 
be an important component of an individual's test score and seems amenable to the 
type of instructional program observed in specific courses in the schools. It 
follows, therefore, that (1) an attempt: should be made to identify students who 
are low in tes t -wiseness , and (2) such students should then be instructed in the 
aspects of test wiseness previously described. The objective of such a learning 
program would be to decrease (1) some of the error of measurement resulting from 
the influence of test wiseness on test scores and (2) the handicap under which 
many examinees apparently operate. 

Once a learning program in test wiseness has been developed, periodic practice 
and/or instruction in standardized and teacher made test-taking techniques for 
all grades would lend more credence, both educational and moral, to the practice 
of making decisions regarding an individual according to scores on group 
administered tests. 

The questions now raised may be (1) fro whom should the learning program be 

f 

designed, (2) what format should it take, (3) who should be responsible for imple- 
menting the program, and (4) what might be the expected outcome. 

Target Audience 

In that the most widely used ETS tents are administered during the secondary 
school years and beyond, it seems reasonable to expect that a learning program 
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developed by ETS should be geared toward students at that level. It should not 
be considered, howevar, that younge** children will not benefit as well from 
such an effort. The various strategies, principles, and related topics discussed 
throughout the paper appear to be applicable to a variety of testing situations 
(e.g. strategies for multiple-choice and analogy items, advisability of changing 
answers, etc.) and would seem to be adaptable to several different tests, if 
neeesr>ary. ' 



Suggested Format 

Although most of the test-wiseness instructional programs cited in the 
literature have taken the form of linear and branching programmed texts, this 
medium need not be the only approach that could be developed with ETS resources. 
However, since these texts are available, appropriate use can and should be ^ 
made of them. 

In an earlier effect, CEEB published a SAT Preparation Course with audio-cassette. 

i 

It is being suggested that the various aspects of test-wiseness, outlined in this 

paper, be discussed extensively and programmatically on cassettes. Rather than 

present the content of the tape as a formal lecture, thought should be given to 

presenting the material in a manner Interesting to young people (perhaps a hip 

dialogue between two persons) so as to maintain the attention of secondary school 

age audiences. Illustrative work materials should accompany the tape and thereby 

actively engage students in the learning program (For example/ the tape recording could 

present the test-wiseness principle "always guess even if the usual correction 

or a more severe penalty for guessing is employed, whenever elimination of options 

provides sufficient chance of profiting 11 ). The accompanying materials would present 

items such that students were required to learn to discriminate when (options can be 

eliminated) and when not (all options are plausible) to guess, 
9 
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Video j^apes are another form in which the test-wiseness learning program 
can be presented. Once the content of the program has been developed, a video 
tape of a teacher in a classroom presenting several lessons in test-wiseness 
cculd be made. This tape would be useful to students who wish to "boost" 
their test-wiseness score, or to pre-prof essional or in-service teachers to 
acquaint them to the topic of test-wiseness. 

Kodak slides which capture -he various aspects and principles of test-wiseness 
are additional devices thai could be useful to teachers in classroom settings or 
to school counselors. The slides could flash on the screen the pronciples (e.g. 
eliminate these options which include specific determiners) and then several 
examples of this principle, in operation, could be presented to the students. 
The student might then be asked to' write an item he felt would illustrate this 
princ iple . 

Finally student handbooks could be distributed to a class of learners. Both 
the student handbook, and a guidebook to be used by classroom teachers or guidance 
counselors, might be developed tq supplement each other, much as regular teacher 
and students texts are designed. * - 

\ ' 

In addition to presenting the principles and strategies previously outlined 

( 

on pages 9 and 10 empirical evidence and other programs in test wiseness that 
have beem implemented, support the following additional topics which might be 
incorporated in a student handbook. 

Implementation 

As might have been implied by the preceding section, a test wiseness program 
could be designed to be implemented (1) by teachers who wish to incorporate 



instruct inn in test wisoness into their regular classroom curriculum (2) by 
counselors in individual or small group guidance sessions prior to a major 
exam, (3) by individual students who wish to become more familiar with the art of 
ta.king tests, or (A) any combination of the ">bove. 

Sugge_s ted Co ntent for Student Handbook 



Based upon empirical research, the following i tents should be incorporate, 
in a student handbook. 



1. Practice tasks intended to permit familiarization with items under a 
variety of conditions: 

a. sentence completion 

b. antonyms 

c. analogies 

d. reading comprehension 

2. Information as to why the test is administered. 

3. Factors involved in scoring tests. 

4. Explanation of test norms and percentile ranking (illustrate to 
students that with as few as three or four additional correct 
responses, one's percentile rank may increase as much as 10 points 
or more) . 

5. Discussion of what the test results will be used for, 

6. Rationale of scpring formulas. 

7. Examples of simulated directions similar to those the applicant 
will face in formal testing. 

a. instruction in comprehending various oral and written directions, 

V 

8. Instruction in marking answers on separate answer sheets. 

9. Simulated tests (discontinued test forms) in which the candidate 
becomes aware of working against time. 
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